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Introduction 

In this paper we present a formula for the number of isomorphism classes of p-rank 
zero etale double covers of genus 2 curves over an algebraically closed field of characteristic 
p > 2. The formula is a byproduct of our search for complete subvarieties of moduli spaces 
of curves. Many moduli spaces are not complete because the objects that they parametrize 
can degenerate. Examples are the moduli spaces of polarized abelian varieties and the 
moduli spaces of curves. For a given quasi-projective variety knowledge of the maximum 
dimension of a complete subvariety gives information on its geometry and cohomology and 
it is a kind of measure how far the variety in question is from being affine. Moreover, in 
the case of a moduli space it tells us when complete families must degenerate. 

For the moduli space A4 g (E)k of curves of genus g > 2 over a field k one has the upper 
bound g — 2 for the dimension of a complete subvariety due to Diaz in characteristic and 
Looijenga in general, but it is not known how good this bound is. For the moduli space 
A g <8> k of principally polarized abelian varieties of dimension g over k one has an upper 
bound for the dimension of a complete subvariety (dim < g(g — l)/2) and it is known 
that this bound is sharp if the characteristic of k is positive. Recently, Keel and Sadun 
[KS, Cor. 1.2] proved that in char. the bound is not sharp for g > 3. In characteristic 
p > the bound is attained by the complete subvariety of abelian varieties of p-rank 
0. This suggests to look at the moduli space of curves in positive characteristic and to 
impose conditions on the p-rank of their Jacobians in order to find complete subvarieties. 
In characteristic p > 2 we get a subvariety of M. g ® k by considering curves C of genus 
g together with an etale double cover C whose p-rank is 0; its expected dimension is 
g — 2. Unfortunately, this doesn't yield a complete subvariety in general, because there 
are components that are not complete. We hope that for almost all primes p there are 
complete components and then these are necessarily of dimension g — 2 and show that 
Diaz's bound is sharp. This necessitates a careful analysis of the components which we 
haven't carried out yet. We did the analysis for genus 2, where we have a formula for the 
number of pairs (C, C) for which the p-rank of C is zero. The number is zero for p = 3, 
but positive for p > 5. One of our main tools is the use of tautological classes. 

The contents of this paper are as follows. We prove that in characteristic p > the 
locus of stable curves of p-rank < / is pure of codimension g — f in M. g ® k. Then we 
consider the Prym map and analyze it using tautological classes. We study the locus of 
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curves with an etale double cover of p-rank in some detail. In particular, in genus 2 we 
obtain a formula for the number of such curves. We end with several examples illustrating 
our formula. 

We shall use the following notation throughout this paper. For nonnegative integers 
g and n such that 2g — 2 + n > 0, we denote by M. g , n the moduli space of n-pointed 
curves of genus g : by M g , n the Deligne-Mumford compactification of -M S)TJ ,, and by M g , n 
the moduli space of stable n-pointed curves of genus g whose dual graph is a tree; often 
these curves are referred to as curves of compact type. All these moduli spaces are smooth 
algebraic stacks defined over Z. We recall that on_an n-pointed curve the n points are 
distinct, ordered, nonsingular points. We can view M g , n as an open substack of M. g , n and 
M gj7l as one of M g , n - When n = (thus g > 2), we write M g , M g , and M g . We have 
M g = M g U U^q 2 ' Aj , where A denotes the divisor of irreducible nodal curves and their 
degenerations and Aj for i > denotes the divisor of reducible curves with components of 
genus i and g — i, and their degenerations. Moreover, we have M g — M g = Aq . By A g 
we denote the moduli space of principally polarized abelian varieties of dimension g and 
by A* the Satake compactification constructed by Faltings. 

In the following we shall work over an algebraically closed field k and we shall often 
write M g (M gi M g: etc.) instead of M g <E> k (M g <E> k, M g ® k, etc.). Since the stacks 
Mg ;n and A g are smooth, the codimension of an intersection is at most the sum of the 
codimensions. We use this fact without comment in §2 and §4. 

If k is of characteristic 7^ 2 then we denote by RM g (resp. RM g ) the moduli space 
of pairs (C, C) of connected etale double covers C — > C of a genus g curve C, with 
C nonsingular (resp. of compact type). The curves C have genus 2g — 1. Such covers 
correspond exactly to points of order 2 in the Jacobian of C. 

§1. Complete subvarieties and tautological classes 

It is a well-known result of Diaz [Dl, Thm.4] that a complete subvariety of the moduli 
space M g <S> k of curves of genus g > 2 has dimension < g — 2 (the extension to positive 
characteristic is due to Looijenga [L, p.412]). But for g > 4 it is not known whether 
this bound is sharp. A similar question occurs for the moduli space A g <S> k of principally 
polarized abelian varieties of dimension g. It is known by [G, Cor. 2. 7] that a complete 
subvariety of A g ® k has codimension at least g (we recall the proof below) . However, here 
one knows that in characteristic p > the moduli space A g <g> F p contains a complete codi- 
mension g subvariety, namely the locus of abelian varieties of p-rank zero, cf. [Ko, Thm.7]. 
As to characteristic 0, Keel and Sadun [KS, Cor. 1.2] recently showed that A g <8> C does 
not contain a codimension g complete subvariety for g > 3, as had been conjectured by 
Oort [03, 2.3G]. Thus the maximum dimension of a complete subvariety of A g ®k depends 
on the characteristic of k ! (It is not known what the maximum dimension of a complete 
subvariety of A g <E> C is for g > 3.) 

Recall the definition of the tautological ring associated to A g . The moduli space A g 
carries a tautological bundle, the Hodge bundle E. It is defined by specifying for every 
principally polarized abelian scheme X over S with zero-section s a rank g vector bundle on 
S: • T ne Chern classes A; = Cj(E) of this bundle are called tautological classes and 
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they generate a Q-subalgebra T*(A g ) of the Chow ring A*(A g ) ®Q, called the tautological 
subring. 

It is shown in [G, Thm.2.1] that the relations ch 2fc (E) = hold VJfe > 1 in T*(A g ), 
hence T*(A g ) is a quotient of T g , the cohomology ring of the compact dual of the Siegel 
upper half space: 

T g = Q[Ai, . . . , A 9 ]/(ch 2 (E), . . . , ch 2s (E)) . 

Note that the vanishing of the first g even ch 2 fc implies the vanishing of all even ch 2 fc (and 
in fact also the vanishing of the odd ch 2 fc+i for all k > g). 

Hence T g is a complete intersection ring with socle in degree ( 5, " 2 ~ 1 ). In [G, Prop. 2. 2] 
it is proved that X g = in T*(A g ), so T*(A g ) is a quotient of T g _\ . It then follows that 
there are no further relations: combine the fact that the p-rank zero locus Vq in A g <8> F p 
is a complete subvariety of pure codimension g (dimension (^)) with the ampleness of Ai 

on A g <S> F p due to Moret-Bailly [M-B, p. 181] (so A^ 2 ^ in every characteristic, and a 
generator of the socle of doesn't vanish on A g ). To summarize the above: 

T*(A g ) = Tg-i . 

( 9 )+i 

This implies that A^ = and hence that a complete subvariety of A g has codimension 
at least g. 

For a more complete discussion see [FL, §11] and [GO, §3], but note that Thm. 3.4 in 
[GO] is at present only a conjecture, unfortunately. 

§2. Subvarieties of the moduli space of curves defined by the p-rank 

We are interested in the loci of curves whose p-rank is 0. For a smooth curve C of 
genus g over an algebraically closed field k of characteristic p > we define the p-rank / 
of C as the p-rank of its Jacobian J: if J[p] denotes the kernel of multiplication by p on J 
then 

#J\p](k)=pf. 

We have < / < g. Alternative definitions of this p-rank are: the semisimple rank of the 
cr-linear operator F induced by the relative Frobenius on H 1 (J,Oj) (or on -ff 1 (C, Oc))', 
or dually, the semisimple rank of the a _1 -linear Cartier operator V on H°(J,Q'j) (or on 

//v\o;.)). 

To extend this definition to stable curves we consider a family C — > B of stable curves 
of genus g over an irreducible base scheme B. We let ivq/b be the relative dualizing sheaf 
and write E = h*(u) C /b) for the Hodge bundle, a locally free sheaf of rank g. On it we 
have the relative Cartier operator V : E — > E. For a point b G B the p-rank of the fibre Cf, 
is defined as the semisimple rank of V on the fibre E&. Equivalently, we can look at the 
action of the relative Frobenius on B}it*Oc- A result of Grothendieck (see [Ka, Th.2.3.1]) 
says that the p-rank is lower semi-continuous on B. 

The following lemma is a variant of a lemma of Oort, see [01, 1.6]. The proof is 
similar to that of Oort and is omitted. 
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(2.1) Lemma. Let B be an irreducible scheme over k and C — > B a stable curve over 
B. Let f be the p-rank of the generic fibre C v . Let W C B denote the closed set over 
which the p-rank of the fibre is at most f — 1. Then either W is empty or W is pure of 
codimension 1 in B. □ 

(2.2) Definition. We define the locus Vf(Ai g ) as the locus of curves with p-rank < / in 
Ai g ; similarly, we define Vf(RAi g ) as the locus of pairs (C',C) for which the p-rank of 
C is < /. We denote by RVo(A4 g ) the locus of pairs (C, C) for which the p-rank of C 
is 0. Similarly, we denote by Vf(A g ) the locus of principally polarized abelian varieties of 
dimension g with p-rank < /. 

We have the following result. 

(2.3) Theorem. The locus Vf(Ai g ) is pure of codimension g — f in M. g . 

Proof. We apply Lemma 2.1 just as in [Ko, p. 164]: Fix r < g. Let C r be any component 
of V r (M. g ). Let C r+ i be a component of V r+ i(A4 g ) containing C r . Note that a priori it is 
possible that C r = C r +i- We obtain a sequence C r C C r+ i C...CC 9 = M g . The lemma 
tells us: for r' = r, r + 1, . . . , g — 1, if C r > ^ C r / + i, then C r > has codimension one in C T , j r \. 
We conclude: the codimension of C r is at most g — r. 

Now let r = 0. As is well-known, Co is a complete subvariety of M. g (the p-rank of a 
generalized abelian variety is at least its torus rank). On the other hand, we have Diaz's 
upper bound 2g — 3 for the dimension of such a complete subvariety; see Lemma 2.4 below. 
It follows that Co has codimension g: the case / = of the theorem. 

If every C r would contain a Co, we would now be done. Since we don't know how 
to establish this, we proceed differently. We prove that every C r has codimension g — r 
in Ai g by induction on r. The case r = has been established, so assume r > 0. Since 
codim(C r ) < g — r < g — 1, the component C r cannot be complete in M. g , hence intersects 
A . The intersection consists of points corresponding to curves of genus g — 1 and p- 
rank < r — 1 with two points identified. The dimension of the intersection is at most 
2{g — 1) — 3 + (r — 1) + 2 = 2g — 4 + r: this is obvious for g = 2 and follows for g > 3 by 
induction. Hence the dimension of C r is 2g — 3 + r. □ 

Remark. It is not clear whether Vf(A g ) intersects the Torelli image of M. g transversally. 
Also, is for example Vo(A4 g ) irreducible for g > 4 ? 

The theorem we just established is the analogue for the moduli space of curves of a 
result of Koblitz [Ko, Thm. 7] for the moduli of principally polarized abelian varieties and 
of the main result of Norman and Oort [NO, Thm. 4.1] for abelian varieties with arbitrary 
polarization. 

The following result was already observed by Diaz [D2, p. 80]. 

(2.4) Lemma. The dimension of a complete subvariety of M 9jn is at most 2g — 3 + n. 

Proof. This is trivial for g < 1. For g > 2, it suffices to prove this for n = 0. By the Diaz- 
Looijenga bound g — 2 for the dimension of a complete subvariety of M. g , we may assume 
that the complete subvariety Y meets some A i? with < i < g/2. Then by induction 
dimy < 1 + (2i -2) + (2(0 -i) -2) = 2^-3. □ 
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(2.5) Lemma. Let X be a complete subvariety of M g of dimension 2g — 3. Then X 
contains points corresponding to chains of g elliptic curves. In particular, X intersects 
every component of the boundary M. g — M. g . 

Proof. Since X is of maximal dimension, the inequalities in the proof of Lemma 2.4 are 
equalities. Some geometric consequences of this will be exploited here. We prove the 
statement by induction on g; the case g = 2 is clear, so assume g > 3. Wejtnow that X 
meets A* for some i with < i < g/2. The inverse image of X n Aj in Mi,i x M g -i,i 
is pure of dimension 2g — 4. Let Z be a component. Let A^i be the projection on the 
first factor and let be the projection on the second factor. Then dimA^i = 2i — 2 

and dimBg-i^i = 2(g — i) — 2. It follows that Z = A^i x B g -i : \. When i = 1, the factor 
Ai^i corresponds to a single elliptic curve. The image B g -i in A4 g -i and, for i > 2, the 
images and B g _i in .M^ resp. M. g -i contain points corresponding to smooth curves; 
this follows again from maximality. Hence A it i is the inverse image of Ai for i > 2 and 
Bg-i^i is the inverse image of B g -i for all i. Now use induction: pick points in Ai (for 
i > 2) and B g _i corresponding to chains of elliptic curves, and (for any i) pick points in 
i and corresponding to marked points on extremal components of the chains. 

Gluing gives a chain of g elliptic curves. □ 

Remark. The proof above is very similar to the way^Keel and Sadun deduce from 
[KS, Cor. 1.2] that a complete X of dimension 2g — 3 in M g doesn't exist in characteristic 
for g > 3. 

Let 7i g be the hyperelliptic locus in M g and H g the hyperelliptic locus in M g , that 
is, the closure of H g in M g . In what follows, we assume that the characteristic is different 
from 2. Then the theory of admissible covers [HM, §4] can be used to describe the stable 
hyperelliptic curves. 

For g > 1, let (D, b±, . . . , 623+2) be a stable (2(7 + 2)-pointed curve of genus 0. The dual 
graph of D is a tree. Define a node of D to be even (resp. odd) if the number of marked 
points on either side of the node is even (resp. odd). There exists a unique admissible 
cover A of degree 2 of D. The inverse image of a component of D is the unique double 
cover of that component that is ramified exactly over the marked points and the odd 
nodes on that component. (The double cover is disconnected exactly when the component 
contains no marked points and only even nodes.) The 2g + 2 Weierstrass points on A are 
by definition the inverse images of the 2g + 2 marked points. The admissible cover is a 
nodal curve. It may contain smooth rational components that meet the rest of the curve 
in only 2 points. (Observe that these necessarily cover a component of D with 2 marked 
points and 1 node.) A stable hyperelliptic curve of genus g > 2 is by definition the stable 
curve of genus g obtained from the admissible cover by contracting those components. 
The Weierstrass points on a stable hyperelliptic curve are the images of the Weierstrass 
points on the admissible cover. Only the Weierstrass points on contracted components 
become singular points. When g = 1, we remember the Weierstrass points and consider 
the admissible covers as stable 4-pointed curves of genus 1. 

A stable hyperelliptic curve is of compact type if and only if all the nodes of the stable 
(2g + 2)-pointed curve D of genus are odd. No components are contracted and these 
curves may be identified with the admissible covers. Therefore H g: considered as a coarse 
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moduli space, may be identified for g > 1 with the quotient of an open set in 

(the complement of the 'even' boundary divisors) by the natural action of the symmetric 

group on 2(7 + 2 letters. 

(2.6) Lemma. (Char. ^ 2.) A complete subvariety of Tig has dimension at most g — 1. 
If Z is a complete subvariety of dimension g — 1 then Z fl Ti g ^ 0. 

Proof. By induction on g, the case g = 1 being trivial. Assume g > 1 and let Z be 
a positive-dimensional complete subvariety of 7i g . Then Z fl Aj 7^ for some i with 
< i < (7/2, since 7Y g is affine. Denote by VF7ij C the locus where the marked point 
is a Weierstrass point. There exists a finite and surjective map WTii x WH g -i — > Aj fl7i g . 
Hence dimZ < 1 + {% — 1) + (g — i — 1) = g — 1. If equality holds then Z is not contained 
in the boundary. □ 

In characteristic a complete subvariety of M g of dimension 2g — 3 doesn't exist for 
g > 3. In positive characteristic we have the following. 

(2.7) Proposition. (Char. > 2.) Let X be a complete subvariety of M g of maximal 
dimension 2g — 3, e.g., an irreducible component of Vo(A4 g ). 

(i) X contains points corresponding to chains of g elliptic curves such that on every non- 
extremal elliptic curve in the chain the two attachment points differ by a 2-torsion 
point. 

(ii) X contains points corresponding to smooth hyperelliptic curves. 

Proof. The proof of the first statement is entirely analogous to the proof of Lemma 2.5; the 
only adaptation required is that one should choose points in A^i and corresponding 
to marked points on extremal components of the chains that differ from the attachment 
point by a point of order 2 (such points exist). Fulton ([HM, p. 88]) shows that the curves 
from (i) are stable hyperelliptic curves. Thus X fl H g is complete of dimension at least 
g — 1 and nonempty. We conclude by applying Lemma 2.6. □ 

§3. Relations between the tautological classes 

We work over an algebraically closed field of characteristic 7^ 2. Let as before RM g 
denote the moduli space of pairs (C, C) with C a smooth curve of genus g and C an etale 
double cover of C. We consider the Prym map 

P:RM g ~^Ag-l, (C',C)» P{C',C). 

Here P(C, C) is the Prym variety of C — > C, i.e., the identity component of the kernel of 
the norm map Nm : J(C') — * J(C). The Prym variety comes with a canonical principal 
polarization S, see [Ml, p. 333]. 

The Prym map P extends to a map 

P : RM g — a;_ u (C, c) » p(C, c), 

where A*_ 1 is the Satake compactification and RM g is the moduli space of pairs (C, C) 
with C a stable curve of genus g of compact type and C an etale double cover of C. Then 
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C is a stable curve of genus 2g — l : not necessarily of compact type. The Prym variety is 
then the identity component of the kernel of the norm map on the generalized Jacobians. 

We establish a relation between the Chern classes of the Hodge bundle and the pull- 
back of such classes under the Prym^map. Consider the universal morphism^of curves 
q : C — > C over the moduli space RM g . The universal curve tx : C = C g — > RM g carries 
a torsion line bundle L defined by q*(Oc) = Oc ® L and L® 2 ~ Oc- 

Let lo be the relative dualizing sheaf of C g over RM g . We define the Prym-Hodge 
bundle E' on RM g by 

E' := 7r*(w <g> L). 

This is a vector bundle of rank g — 1 on RM g . We denote its Chern classes by A - = Cj(E'), 
for 1 < i < g — 1. 

(3.1) Theorem. Let <$> : RM g -> .M g be tie map (C, C) i-> C. Then ch(E') = 
ch(0*(E))-l. 

Proof. Applying the Grothendieck-Riemann-Roch theorem (GRR) to the line bundle uj®L 
and the morphism it gives 

ch(E') = 7r*(ch(w <g> L) • Td v (u)). 
Since we work in Chow groups tensored with Q we can disregard torsion classes and we 



may replace the term u> <8> L on the right by u, so that 

ch(E') = n^ch(uj) ■ Td v (w)). (1) 

But applying GRR to u and 7r gives 

ch(7riw) = ch(0*(E)) - 1 = 7r*(ch(w) • Td v (w)), (2) 

so that by comparing (1) and (2) we find ch(E') = ch(0*(E)) — 1 as required. □ 

(3.2) Corollary. We have 0*(A<) = A^ for i = 1, . . . ,g - 1. □ 



Note that it also follows that A s vanishes in the Chow ring A* (A4 g ). This is compatible 
with the fact that it vanishes on A g . 

Denote by V : RM g -> M 2g -i the morphism (C',C) i-> C". Then V*(E 25 -i) = 
0*(E) © E', hence V*(Ai) = 2X[. 

(3.3) Corollary. Tie Toreiii maps RM g -> „4* fresp. iLM ff -> sending (C',C) 

to J(C) (resp. J(C") J are constant on any complete connected algebraic subset of a fiber 
of the extended Prym map P : RM g —* A*_ 1 . 

Proof. Clearly P*(Ai) = X[. This is zero on a fiber of P, hence the same holds for 0*(Ai) 
and ip*(Xi). These are pull-backs of ample classes on A* resp. A\ g _ x via the Torelli maps. 
The result follows. □ 
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(3.4) Corollary. The restriction of the Prym map to a complete subvariety Y of RM g 
is quasi- finite on Y n RM g . □ 



§4. The locus V (RM g ) 

We now consider in RAi g the locus Vq(RM. 9 ) of etale double covers (C, C) where the 
p-rank of C is zero (p > 2). Here is our motivation for studying this locus. Consider A g 
and a toroidal compactification A g of the type constructed in [FC, Ch.IV] (such that the 
Hodge bundle extends). The class X g has zero intersection with all boundary classes (those 
from A g —A g ). If a positive multiple of X g can be written as an effective sum of subvarieties 
not contained in the boundary, then all those subvarieties are complete subvarieties of A g 
of codimension g. The locus Vo is a (canonical) effective cycle of this type in positive 
characteristic [G, Thm.9.2], while in characteristic such an effective cycle doesn't exist 
for g > 3 [KS, Cor. 1.2]. As we saw in §2, the situation for M g and its compactification 
M g is exactly analogous. 

Now consider Ai g , with compactification M. g . The class X g X g -i has zero intersection 
with all boundary classes [F, p. 112] and appears to be the natural analogue of the class 
X g for A g and A4 g (cf. [FP, §0]). In the search for an effective representative of a multiple 
of XgXg-i, it seems natural to combine the p-rank zero requirement with a condition 
specific to curves. Certainly, the construction of the Prym variety P(C , C) associated 
to an etale double cover C of a curve C plays an important role in the theory of curves 
and Jacobians. The Prym variety is a principally polarized abelian variety of dimension 
g(C) — 1. The requirement that it have p-rank zero should then impose the desired g — 1 
additional conditions; or equivalently, one may combine the two requirements by asking 
that C have p-rank zero. Note also that the class of Vo(M.2 g -i) is a multiple of A23-1 and 
that ip*(\2 9 -i) = 4>*(XgXg-i). This explains our interest in the locus V (RM g ). 

(4.1) Proposition. Every irreducible component of V (RM g ) has dimension at least 
g — 2. A complete irreducible component ofV (RM g ) has dimension equal to g — 2. 

Proof The first statement follows by applying Lemma 2.1 to the family of curves C over 
RMg. The second statement follows then from Diaz-Looijenga. □ 

Let as before ij> : RM g — > M.2g-i be the morphism (C, C) 1— > C . We define 

Vo(RMg) =ii- 1 (V (M 2g -i)). 

Clearly, this contains Vq{RM. 9 ) = ^ _1 (Vo(A^2g-i)), and every irreducible component of 
V (RMg) has dimension at least g — 2, by Lemma 2.1 again. Observe that V (RM g ) is 
complete: it is a closed sublocus of the complete locus RV (M g ). 

For g = 2, all components of Vo(RA4 g ) have the expected dimension (7 — 2 = 0. In 
fact, from Corollary 3.3 and the identification M.2 = A 2 , the Prym map P is finite on 
RVq{M.2)- In §6 we count the number of points of V^-RA^)- It turns out that Vq{RM.2) 
is empty for p = 3, but nonempty for p > 5. 

For g > 3, the situation is more complicated. We first analyze the_case g = 3 in some 
detail. We have the following description of the components of Vq(RM^). 
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(4.2) Proposition. 

(i) All components of Vo(RAis) have dimension equal to g — 2 = 1. 

(ii) For every pair (£)', D) G Vo(-RA^2) an d every supersingular elliptic curve E, there is 
a component of Vq(RM.z) isomorphic to D. The curve C p corresponding to p G D is 
obtained by gluing D and E at p, while C' p is obtained by gluing two copies of E to 
D' at the two inverse images of p. 

(in) For every pair (E',E) G Vq{RM.\,\) and every component X of Vq{M.2), there is a 
component of ' Vq{RM.z) isomorphic to the universal curve over X (considered as a 
subvariety of A4 2,1 )■ The curve C corresponding to D G X and p G D consists of E 
and D glued at p, while C is obtained by gluing two copies of D to E' (the inverse 
images ofOeE are identified with the two points p). 

(iv) For every pair (£", E) G Vq(RM.\,\) and every two supersingular elliptic curves F and 
G, there is a component of Vq{RM.z) isomorphic to E (considered as a subvariety of 
M.\^)- The curve C p corresponding to 7^ p G E consists of E with F and G glued 
on at and p, while C' p consists of E' with two copies of both F and G glued on at 
the respective inverse images. 

The components of type (ii)-(iv) are the components of Vq(RM.^) not intersecting RM3. 




G 




(ii) (Hi) (iv) 

(Figures (iii) and (iv) represent general points of the corresponding components.) 

Proof. From Corollary 3.4, the restriction of the Prym map to RVo(Ai 9 ) is quasi-finite. 
For (7 = 3 this means that all components of Vq{RM.z) have dimension equal to g — 2 = 1, 
the dimension of V (A2). By taking the closure, we obtain the components of Vo(RMs) 
intersecting RM3. It is straightforward to check that (ii), (iii), and (iv) give a complete 
description of the p-rank zero etale double covers of reducible curves o^genus 3. Since these 
families are positive-dimensional, they yield the components of Vq(RM.3) not intersecting 
RM 3 . □ 

The components of type (iii) have dimension 2 and provide the first examples of 
components of Vo(RAi g ) of dimension greater than g — 2. Taking X to be a component 
of Vo(A4 g -i), we find components of Vo(RA4 g ) of dimension 2g — 4. 

Return to genus 3. From the description in Proposition 4.2, one might expect that all 
components of Vq(RM3) are complete. However, this is not the case, as is shown by the 
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following argument that we learned from Oort. 

(4.3) Proposition. (Oort.) If V (RM g -i) is nonempty, then it has a component of 
dimension > g — 3 (necessarily noncomplete) or V (RM g ) has a noncomplete component. 

Proof. Consider ifi : RM g — > M.2 g -i and the Torelli map t : M.2 g -i — > ^L-i- The image 
tip(RMg) has dimension 3g — 3; every component of the intersection with Vo(-4.2g-i) has 
dimension at least g — 2. The intersection Vo(A 2g -i) H tip(RM g ) equals t-i/>(Vb(-R.M ff )). 

Let F be a component of Vq(RM. 9 -i). We may assume that dimF = (7 — 3. Consider 
the following family in Vo(RA4 g ) isomorphic to the universal curve over (p(Y): for (D', D) e 
y and p G -D, construct C by gluing a super singular elliptic curve E to D at p and C" 
by gluing two copies of E to D' (just as in type (ii) above). Denote this family by Z. 
Then tip(Z) has dimension g — 3. Therefore it is strictly contained in a component of 
^(V^irLMg)). That component necessarily contains Jacobians of smooth curves C and 
yields a noncomplete component of Vq(RM. 9 ). □ 

We may similarly consider a family of pairs (C, C), where C consists of two smooth 
components, D of genus i and E 1 of genus j, and C" consists of D' of genus 2i — 1 and 
two copies of -E. Let (Z)', D) vary in a component Y of Vo(-RTWi) and i? in a component 
of Vo(yWj). Since the attachment points vary also, the dimension of this family equals 
(dimF + 1) + (2j - 2) > (i - 1) + (2j - 2) = j + # - 3. (Mutatis mutandis, this includes 
the cases i = 1 and j = 1.) Denoting the family by Z, we have 



dim ttjj(Z) = 



'0, 


i = l, 


j 


= l; 


^ j+g — 4 = 2g — 5, 


i = l, 


j 


> l; 


dimY > i - 2 = g - 3, 


z > 1, 


j 


= l; 


k dim y + 2j - 3 > j + fir - 5, 


% > 1, 


j 


> 1. 



The proposition uses the case j = 1 for g > 3. We see that for g > 4 one may also use the 
case j = 2. This proves: if Vo(RJA g -2) is nonempty, then it has a component of excess 
dimension or Vo(RAi g ) has a noncomplete component. 

(4.4) Corollary. For p > 5, the locus Vq(RM. 9 ) has a noncomplete component for g = 3, 
4, and 5. 

Proof. Use that Vb(i?A^2) is nonempty and of dimension 0. Then Vq(RA4s) has a non- 
complete component. It has dimension 1. Now use the proposition again for g = 4 and 
the discussion above for g = 5. □ 

We study the case p = 3 and g = 3 in the next section. 

§5. The locus V (RM 3 ) for p = 3 

We start by recalling some facts concerning hyperelliptic curves. Assume char(/c) 7^ 2. 
Every etale double cover C of a hyperelliptic curve C arises from splitting up the set B 
of branch points of C into two disjoint subsets of even cardinality: 

B = B 1 UB 2 . 
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Then C is a Galois cover of P 1 with Galois group Z/2 x Z/2 and we have a diagram 



C" 

/ I \ 
C\ C C 2 

\ I / 

pi 

where C\ (resp. C2) is the hyperelliptic curve with branch points B\ (resp. _B 2 ). Moreover, 
the Prym variety of C over C is isomorphic to J{C\) x J(C2) (cf. [Ml, p. 346]). Note that 
C is hyperelliptic if and only if C\ or C 2 is a rational curve. 

Next, assume that k has characteristic p > 2. The p-rank of a hyperelliptic curve 
C can be computed easily. If C is given by y 2 = f(x) with f(x) a polynomial without 
multiple roots of degree 2g + 1 or 2g + 2, then the regular differentials x l ~ 1 dx/y with 
1 < * < fi 1 form a basis. Write 

00 

f(x) i - p - 1)/2 = a{x) = ^a l x\ 
then the Hasse-Witt matrix with respect to this basis is 

/ a p -i Cl2p-1 ■ ■ ■ CLgp-l \ 
a p _ 2 a 2p -2 • • • 

\a p _ S a 2 p- S ... Cigp-g / 

(Cf. [SV, p. 54].) The p-rank of C equals the semisimple rank of H. In particular, C has 
p-rank if and only if 

H-H<*) .....H** 9 ' 1 ) =0, 

where H^ p is the matrix obtained from H by raising every entry to the (p*)th power. 
This is independent of the chosen basis. 

Let now p = 3. 

(5.1) Lemma. A hyperelliptic curve C of genus 5 with a Gxed-point-free involution 
cannot have 3-rank 0. 

Proof. Let C be the quotient curve of genus 3. Then C is hyperelliptic and C is an etale 
double cover of C. By the discussion above, the Prym variety of (C, C) is the Jacobian 
of a curve D of genus 2. Assume that C has 3-rank 0. Then D has 3-rank 0. Write D as 
y 2 = X)i=o a i xi - The Hasse-Witt matrix of D equals 

yai a 4 y 
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We may assume a 5 = 1 and a 4 = 0. It follows then that a-y = a 2 = 0. We may assume 
ao = 1 and write D as 

y 2 = x 5 + ax 3 + 1. 

Then C can be written as 

y 2 = (x 2 + bx + c) (x 5 + ax 3 + 1) = x 7 + bx 6 + (a + c)x 5 + abx 4 + acx 3 + x 2 + bx + c 
and the Hasse-Witt matrix of C equals 

(l a + c 0\ 
H c = 6 a6 1 , 
y c ac b J 

with determinant c 2 — cb 2 = c(c — 6 2 ). The discriminant of x 2 + bx + c equals b 2 — c, so it 
follows that c = 0. Then 

/l a 0\ 
£Tc = & a6 1 
\0 bj 

and the (3, 3) -entry of H ■ # (3) • # (9) equals 6 13 . This forces b = and we obtain a 
contradiction. □ 

(5.2) Proposition. For p = 3, the locus V (RMs) doesn't have a complete component. 

Proof. Suppose instead that there is a complete one-dimensional family X of smooth 
curves of genus 5, varying in moduli, such that each curve has 3-rank and possesses 
a fixed-point-free involution. On X the degree of Ai is positive. Hence the degree of 
the divisor T in A4 5 of trigonal curves and their degenerations is positive, since [T] = 8Ai 
(cf. [HM, p.24]). It is easy to see that a trigonal curve of genus 5 cannot have a fixed-point- 
free involution: the #3 is unique, thus fixed by the involution; there are 2 fixed divisors; 
this forces fixed points on the curve. The intersection points of X and T come therefore 
from degenerate trigonal curves where the trigonal system decomposes as a g\ plus a base 
point. These curves are hyperelliptic and Lemma 5.1 gives a contradiction. □ 

(5.3) Example. For p = 5, there do exist smooth hyperelliptic curves of genus 5 with 
5-rank and possessing a fixed-point-free involution. Consider C of genus 3 given by 

y 2 = (x 2 - x + 2)(x 5 + x 4 + 2x 2 - 2x) = x r + x 5 - x 4 + x 3 + x 2 + x. 

One easily checks that both C and D, given by y 2 = f(x) = x 5 + x 4 + 2x 2 — 2x, have 
5-rank 0. The etale double cover C is given by 

y 2 = ff X ~ 2 V2x-l) 6 = 2x 11 -2x 10 +x 9 + x 8 -x 7 -2x 6 + 2x 5 -x 4 +x 3 -2x 2 -2x + 2, 
with fixed-point-free involution (x, y) 1— > ( 2x-i •> ~v)- 
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(5.4) Example. For p = 3, there do exist smooth hyperelliptic curves of genus 7 with 
3-rank and possessing a fixed-point-free involution. Consider C of genus 4 given by 

y 2 = x(x 8 + x 6 + x 5 - x 3 - x 2 - 1). 

One easily checks that both C and D, given by y 2 = f(x) = x 8 + x 6 + x 5 — x 3 — x 2 — 1, have 
3-rank 0. The etale double cover C is given by y 2 = f(x 2 ) = x 16 + x 12 + x 10 — x 6 — x 4 — 1, 
with fixed-point-free involution (x, y) i— > (—x, —y). 

(5.5) Example. For p = 3, there do exist smooth hyperelliptic curves of genus 3 possessing 
an etale double cover with 3-rank 0. Consider C of genus 3 given by 

y 2 = ( X 3 + X + 1)(X 4 + X - 1) = X 7 + X 5 - X 4 - X 3 + X 2 - 1. 

One easily checks that C and the elliptic curves v 2 = x 3 + x + 1 and w 2 = x 4 + x — 1 have 
3-rank 0. The etale double cover C is the normalization of 

v s _ w e _ v * w 2 _ v z_ w 4_ v * + v 2_ 1 = 0j 

with fixed-point-free involution induced by (v, w) h- > (—v, —w). 

§6. The number of p-rank zero etale double covers of genus 2 curves 

In this section we count the number of points of Vq(RM.2), that is, we count the 
number of pairs (C, C) with C a smooth curve of genus 2 and C an etale double cover of 
C withp-rank zero. Our method is to use intersection theory onjstacks. In the terminology 
of Mumford [M2, p. 293], we intersect the Q-classes of RV Q (M2) and P~ 1 V (Ai) inside 
RM.2- The intersection has dimension zero and the Q-class of every point is counted with 
the intersectionjmdtiplicity in the universal deformation space. This gives the number of 
points of Vq{RM.2) and we need to subtract the contribution of the reducible curves to 
obtain the number of points of Vq{RM.2}- 

(6.1) Theorem. The weighted number of isomorphism classes of pairs (C, C) with C a 
smooth curve of genus 2 and C an etale double cover of C with p-rank zero is given by 

where m(C',C) G Z>i is the intersection multiplicity in the universal deformation space 
and |Aut(C", C)\ denotes the number of automorphisms of C that fix the point of order 2 
in J(C) corresponding to C. 

Proof Since Vq{A\) has class (p — l)Ai, the class of P~ 1 Vo(Ai) equals 

(p - l)P*(Ai) = (p - 1)A; = (p - 1)0* Ax, 
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by Corollary 3.2. By [G, Thm.9.2], the class of RV (M 2 ) equals (p - l)(p 2 - 1)0* A 2 (note 
that M.2 = A2 as stacks). To evaluate the degree of the intersection, we apply 0*. We 
obtain 

M(P - 1)0* Ax • (p - l)(p 2 - 1)0* A 2 ) = 15(p - 1) V - l)AiA 2 = g^(P - 1) V " 1), 

since A1A2 = on M.2- This is the weighted number of points of V G (RM.2)- 

The number of reducible curves in this intersection can be computed easily. We need 
an ordered pair (Ei, E 2 ) of supersingular elliptic curves and a point of order 2 on E\. We 
find 

3-^.^ = —{p-lf. 
24 24 192 KF ' 

In the next proposition we show that these pairs count with multiplicity p + 1. Therefore 
the weighted number of points of V"o(-R.M 2 ) equals 

1 -Op - i)V - 1) - -L(p + i)(p - 1) 2 = -l-(p - i) 2 Cp + i)((p - 1) - 2) 



384^ ' v ^ ' 192 v ^ /vr ' 384 



1 (p-3)(p-l) 2 (p + l), 



384 



as required. □ 
In §7 we give a number of examples illustrating this formula. Note that V (RM.2) is 
empty for p = 3, but nonempty for p > 5. 

(6.2) Proposition. The locus RV (M-2) intersects P _1 V (^li) with multiplicity p + 1 at 
points corresponding to stable curves with two elliptic components. 

Proof. For a smooth etale double cover D' — > D corresponding to a point 77 of order 2 in 
J(.D) the codifferential of the Prym map P : RM. g — > «4 g -i is given by the multiplication 
map Sym 2 (H°(D,u)D © f/)) — > H°(D,uj 2 d ). We extend this result to certain etale double 
covers of reducible curves of genus 2 in the following manner. 

Let C be a stable curve of genus 2 consisting of two elliptic components E\ and E 2 . 
We let Def(C) be the space of infinitesimal deformations of C. Since M.2 = A2, it can be 
identified with the space of infinitesimal deformations of the Jacobian E\ x E2 of C (as a 
principally polarized abelian surface). We thus get 

Def(C) = Sym 2 (T El @T E2 ) = Sym 2 (T El ) © (T El © T Sa ) © Sym 2 (T S2 ), 

where Tg. denotes the tangent space to -Ej at the origin. Note that the tangent space and 
the cotangent space to A2 are identified via the principal polarization. 

Consider now the etale double cover C of C defined by a point rj of order 2 of E\ . It 
consists of the etale double cover E[ of E\ defined by 77 and two copies of E 2 . Then 

H°(C, oo c ®n) = H°(E U oo El © rj) © #°(£ 2 , w E J = tf°(£ 2 , w E J = T E , . 

With these identifications, the codifferential of the Prym map becomes the inclusion 

Sym 2 (T E2 ) -> Sym 2 ^) © (T El © T E2 ) © Sym 2 (T S2 ) 
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and the tangent space to the fiber of the Prym map is given by 



Sym 2 (T El )®(T El ®T E2 ). 

On the other hand, for a C which is supersingular (i.e., both Ei are supersingular) one 
can calculate the directions in the deformation space Def (C) along which supersingularity 
(or equivalently, the condition that the p-rank is 0) is preserved. According to a calculation 
by Oort [02, p. 258] we find the p + 1 directions 



i c 



G Sym 2 (T El ©T E2 ), 



where ( p+1 = — 1. Here the ith coordinate corresponds to T Ei . These tangent vectors are 
not contained in Sjm 2 (T El ) © (T El ®T E2 ). □ 

§7. Examples 

We now give a number of examples to illustrate the formula of the preceding section. In §5 
we discussed how the p-rank of a hyperelliptic curve may be computed and how the etale 
double covers of hyperelliptic curves can be described. In the case where g(C) = 2 we may 
assume that C 2 is a rational curve and that C\ = E is an elliptic curve. The Prym variety 
of (C, C) is isomorphic to E. Below, we will identify the etale double cover C with a 
point i] of order 2 in J(C). With C defined by y 2 = f(x), we give r\ by choosing 2 of the 
6 branch points. 

(7.1) p = 3. Over F3 the elliptic curve with j = = 1728 can be written y 2 = x 3 — x. 
Hence C can be written y 2 = (x 2 + ax + b)(x 3 — x) with r] corresponding to x 2 + ax + b. 
Then the Hasse-Witt matrix equals 



H = 



a 2 a 5 \ _ f -a 1 
ai 04/ I — b a 



and we have the condition H ■ = 0. In particular, = det(H) = b — a 2 ; note that 
this condition already makes C singular: x 2 + ax + a 2 = (x — a) 2 . So, not only there isn't 
a smooth C of genus 2 and 3-rank with unramified double cover C of 3-rank (hence 
Prym of 3-rank 0), there isn't even a smooth C of 3-rank < 1 with Prym of 3-rank 0. 

(7.2) p = 5. Over F5 the elliptic curve with p-rank has j = and can be written 
y 2 = x 3 — 1. Hence C can be written y 2 = (x 2 + ax + b)(x 3 — 1) with r] corresponding to 
x 2 + ax + b. Now a(x) = f(x) 2 and 



H = 



0.4 ag 
a 3 a 8 



One checks that det(H) = gives b = —a 2 or b = — 2/ a. Since b = —a 2 leads to a singular 
curve (x 2 + ax — a 2 = (x — 2a) 2 ) one is left with b = —2/ a. Working out H ■ = leads 
to the necessary condition a 15 — a 9 + 1 = 0; but a 3 + 3 = is to be avoided, as it leads to 
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a singular curve. The residual polynomial in F 5 [a] of degree 12 has 3 irreducible factors 
of degree 4. Now note that the automorphism group of the 4-tuple {1, £, £ 2 , oo} (with £ a 
primitive cube root of 1) is acting on the solution set, reflecting the choice of rr-coordinate 
in writing the elliptic curve with j = as y 2 = x 3 — 1. It is the group A±, generated by the 
transformations x i— > Qx and x t— > (x + 2)/(x — 1). A calculation shows that the solution 
set is one A4-orbit. So there is one isomorphism class of pairs (C, C) with C" of 5-rank 
0; it is defined over F5 and has only the hyperelliptic automorphism y 1— > — y (this follows 
since the A4-orbit has length 12). So the answer to the counting problem is 

1 _ 2-4-4-6 

2 ~ 384 

as in the theorem; this proves that the multiplicity m(C", C) = 1. Finally, one would of 
course like to have a model over F 5 . One can try to take r\ = {0, 00} although this is not 
guaranteed to work: 

y 2 = x(x 4 + b 3 x 3 + b 2 x 2 + b\x + b ) 

with the bi G F5. The Prym condition is just b\ + 26163 + 26o = wheras the condition for 
C simplifies to H 2 = 0. A solution is (for instance): 

y 2 = x (x 4 + x 3 + 2x + 3) with H = 

Remark. The locus RVq{M.2) is nonsingular at a point (C, C) with a-number a(C) = 1, 
while it has multiplicity p + 1 at a point with a(C) = 2. If the intersection with P _1 Vo(v4.i) 
is transversal, the multiplicity m(C, C) equals 1 resp. p + 1. 

(7.3) p = 7. Over F 7 we start with the form y 2 = (x 2 + ax + b)(x 3 — x). To find solutions 
one can proceed as follows. The 4 entries of H-H^ and det(H) are 5 polynomials in a and 
6 that have to vanish simultaneously. Using Maple, one computes the resultants modulo 7 
w.r.t. the variable 6 (resp. a) of the determinant and each of the 4 entries without problem; 
taking the polynomial g.c.d. modulo 7 of the 4 resultants leads to a polynomial in a (resp. 6) 
of degree 65 in both cases. The multiple factors in the a-polynomial are a 19 (a + 2) 3 (a + 5) 3 
reducing the number of possible a- values to 43. The factors of the 6-polynomial all have 
multiplicity at least 2, with (6 + 6) 22 (6 2 + 36 + 1) 4 6 3 the factors with multiplicity > 2, 
reducing the number of possible 6-values to 20. Examining the singular solutions leads 
to the exclusion of a = ±2 and of 6 = 0, which leaves us with 41 possible a-values and 
19 possible 6-values. Clearly, when (a, 6) is a solution, so is (—a, 6); more generally, the 
automorphism group of the 4-tuple {0, 1, — 1, 00}, the group D4, is acting on the solution 
set; the ^-transformations x 1— > — x resp. x 1— > (x — l)/(x+l) have the effect (a, 6) 1— > (—a, 6) 
resp. 

26- 2 6+l-a\ 
6 + l + a'6+l + a/' 

We examine the orbits of this action. It is clear that (a, 6) = (0, 1) is the unique fixed point; 
it is in fact the unique solution with a non-trivial stabilizer. Here C is the curve y 2 = x 5 — x 



i I) 
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which has 48 automorphisms; rj = {i, — i}; the pair (C,rj) has 16 automorphisms. Note 
that a(C) = 2 (since H = 0); hence we expect to count (C,rj) with multiplicity p + 1 = 8 
for a total contribution of 8/16 = 1/2. The pair (C, rj) is isomorphic to (C, {0, oo}). 

It is easy to check that the only solution with a = is b = 1, and vice versa. For the 
solutions with a nonzero, 40 possible a-values remain. These fall into 5 orbits. We find 
only 18 possible 6-values since a 'ramification' occurs: to b 2 + Sb + 1 = correspond 8 
a-values (the roots of (a 4 — a 2 — l)(a 4 — 3a 2 — 1)) instead of the expected 4. 

So we find 5 more solutions (C,rj), each having only the automorphisms y i— > ±y; 
also, a(C) = 1 for all five. Hence each contributes 1/2, so that the answer to the counting 
problem is 



6- 




4-6-6-8 
384 



Two of the 5 -D.4-orbits are given by a polynomial in ¥^[a] of degree 16 (with 4 irreducible 
factors of degree 4). Hence the 2 corresponding isomorphism classes (C, rj) are defined over 
F 49 (a Galois orbit of length 2). With some work, one finds explicit representatives: 



C : y 2 = x(x^ + x 3 + (i + l)x 2 + (Si - 2)x - i) = x(x 2 + Si - 2)(x 2 + x - 2i + 3) 
rj : x 2 + x — 2i + 3 

E : y 2 = x(x 2 + Si - 2) (j = 1728 = -1) 



with i = v— T € F49. 



The other three L>4-orbits are given by a polynomial in ¥^[a] of degree 24 (with 4 
irreducible factors of degree 6). The 3 corresponding isomorphism classes (C, rj) are defined 
over F343 (a Galois orbit of length 3) . One eventually finds explicit representatives: 

(C: y 2 = (x 2 + l)(x 4 + x 3 + (p 2 + p + 2)x 2 + (Sp 2 - Sp + l)x + (hp 2 - Sp + 3)) 
< 77 : x 2 + 1 

[E: y 2 = x 4 + x 3 + (p 2 + p + 2)x 2 + (Sp 2 -Sp+l)x + (5p 2 -Sp + S) (j = 1728) 
with p = \[2 G F343. 

(7.4) p = 11. There are two cases: (A) the Prym variety is the elliptic curve with j = 1728; 
we take y 2 = (x 3 — x)(x 2 + ax + b) as (C, 77); (B) the Prym is the elliptic curve with j = 0; 
we work with y 2 = (x 3 — l)(x 2 + ax + b). 

In case (A) one initially finds a polynomial in a of degree 340; only its 5 linear factors 
come with (varying) multiplicities; the reduced polynomial has degree 197. The initial 
polynomial in b also has degree 340; every factor has at least multiplicity 2, and only its 4 
linear factors have higher multiplicities; the reduced polynomial has degree 100. As it turns 
out, the linear factors lead either to singular curves or to the 4 solutions (a, b) = (0, 2), (0,6), 
(3,1), (8,1). These 4 solutions form a single L>4-orbit; hence the corresponding pair (C,rj) 
has 4 automorphisms. One easily checks that a(C) = 2 so that the expected contribution 
is (p+ l)/4 = 12/4 = 3. The remaining 192 solutions of the reduced polynomial in a form 
24 orbits; their expected contribution is 24/2 = 12, making the total expected contribution 
in case (A) equal to 15. The Galois orbits have lengths 1, 4, 4, 4, 6, and 6. 




17 



In case (B) one initially finds a polynomial in a of degree 270; only a linear and a 
quadratic factor come with a multiplicity; the reduced polynomial has degree 243. Exactly 
the same happens for the polynomial in b. As it turns out, the linear and the quadratic 
factor lead to singular curves. The remaining 240 solutions of the reduced polynomial in a 
form 20 A 4 -orbits; the expected contribution in case (B) is 20/2 = 10. The Galois orbits 
have lengths 3, 3, 4, and 10. 

Hence the total expected contribution equals 25, as in Theorem 6.1. So all intersections 
are transversal. 

(7.5) p = 13. Here we find the first example of a non-transversal intersection. We 
work with y 2 = (x 3 + x + 4)(x 2 + ax + b) with Prym variety the supersingular elliptic 
curve y 2 = x 3 + x + 4. One initially finds a polynomial in a of degree 466; a linear 
and a quadratic factor come with multiplicity 15 and yield singular curves; the factor 
a 4 + 8a 3 + 7a 2 + 8a + 9 comes with multiplicity 2; one linear factor doesn't give a solution; 
the remaining reduced polynomial has degree 416. The remaining polynomial in b has 
degree 416 as well. The group Di acts; if all intersections were transversal and all points of 
intersection had a-number 1, then the total contribution would be 104/2 = 52. However, 
the answer is 105/2 according to Theorem 6.1. The only possible explanation is that 
exactly one of the intersections is non-transversal and has multiplicity 2. Indeed, the 
factor a 4 + 8a 3 + 7a 2 + 8a + 9 yields that intersection point. The pair (C, if) is defined over 
F i3 ; an equation is 

y 2 = x 5 + x 4 + 6x 3 - 2x 2 + 2x 

with rj = {0, oo}. This is a nonsingular point of RVq{M.2) and the tangent line may be 
given as 

y 2 = x 5 + x 4 + (t + 6) x 3 + (2t - 2)x 2 + (2 - t)x 
with the same rj. The moving elliptic curve is 

y 2 = x 4 + x 3 + (t + 6)x 2 + (2t-2)x + 2-t 

and the coefficient of x 12 in (x 4 + x 3 + (t + 6)x 2 + (2t — 2)x + 2 — t) Q is t 2 times a unit, 
which proves the multiplicity statement. 

(7.6) p = 17. There are two supersingular j-invariants. For the elliptic curve y 2 = x 3 +x+l 
we find a Galois orbit of length 2 with a-number 2 and 4 automorphisms, contributing 
2(p+l)/4 = 9, and a pair (C, rj) defined over F i7 giving a simple tangency and contributing 
1. The remaining solutions contribute 928/8 = 116 for a total contribution of 126. 

For the elliptic curve y 2 = x 3 — 1 we find the first example of a non-transversal 
intersection with a-number 2. This proves that the intersection multiplicity cannot be 
read off from the Hasse-Witt matrix. Let the pair (C, rj) be given by 

y 2 = (x 3 -l)(x 2 -2x -2). 

The 18 tangent directions to RVq{M.2) can be written down explicitly. Two of them lie 
in the tangent plane to the Prym fiber and it turns out that they yield simple tangencies. 
The intersection multiplicity equals 20. The A 4 -orbit of this pair has length 3 and it 
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contributes 3-20/24 = 5/2. There is also an ^-orbit of length 12 with a-number 2, 
contributing 12 • 18/24 = 9. The remaining solutions contribute 732/24 = 61/2. The total 
contribution for y 2 = x s — 1 equals 42. 

Adding up the contributions for the two elliptic curves, we obtain 168, as in Theorem 

6.1. 



Acknowledgements. We thank Frans Oort for several useful discussions; Proposition 
4.3 is due to him. We thank the Goran Gustafsson Stiftelse for support of the second 
author's visit to KTH in March 2001. The first author thanks Princeton University and 
the Swedish Research Council for support. Finally, we thank the referee for several useful 
comments. 



References 

[Dl] S. Diaz: A bound on the dimensions of complete subvarieties of Ai g . Duke Math. J. 
51 (1984), 405-408. 

[D2] S. Diaz: Complete subvarieties of the moduli space of smooth curves. In Algebraic 
geometry, Bowdoin 1985 (S. Bloch, ed.), 77-81, Proc. Sympos. Pure Math., 46, Part 1, 
Amer. Math. Soc, Providence, RI, 1987. 

[F] C. Faber: A conjectural description of the tautological ring of the moduli space of 
curves. In Moduli of curves and abelian varieties (C. Faber and E. Looijenga, eds.), 109- 
129, Aspects Math., E33, Vieweg, Braunschweig, 1999. 

[FL] C. Faber and E. Looijenga: Remarks on moduli of curves. In Moduli of curves and 
abelian varieties (C. Faber and E. Looijenga, eds.), 23-45, Aspects Math., E33, Vieweg, 
Braunschweig, 1999. 

[FP] C. Faber and R. Pandharipande: Logarithmic series and Hodge integrals in the tau- 
tological ring. With an appendix by Don Zagier. Michigan Math. J. 48 (2000), 215-252. 
[FC] G. Faltings and C.-L. Chai: Degeneration of abelian varieties. Ergebnisse der Math- 
ematik und ihrer Grenzgebiete 22. Springer- Verlag, Berlin, 1990. xii+316 pp. 

[G] G. van der Geer: Cycles on the moduli space of abelian varieties. In Moduli of curves 
and abelian varieties (C. Faber and E. Looijenga, eds.), 65-89, Aspects Math., E33, Vieweg, 
Braunschweig, 1999. 

[GO] G. van der Geer and F. Oort: Moduli of abelian varieties: a short introduction and 
survey. In Moduli of curves and abelian varieties (C. Faber and E. Looijenga, eds.), 1-21, 
Aspects Math., E33, Vieweg, Braunschweig, 1999. 

[HM] J. Harris and D. Mumford: On the Kodaira dimension of the moduli space of curves. 
With an appendix by William Fulton. Invent. Math. 67 (1982), 23-88. 
[Ka] N. Katz: Slope filtration of F-crystals. Journees de Geometrie Algebrique de Rennes 
(Rennes, 1978), Vol. I, 113-163, Asterisque No. 63 (1979). 

[KS] S. Keel and L. Sadun: Oort's conjecture for A a . Preprint 2002, |math.AG/0204229| , 
to appear in J. Amer. Math. Soc. 

[Ko] N. Koblitz: p-adic variation of the zeta-function over families of varieties defined over 
finite fields. Compositio Math. 31 (1975), 119-218. 

[L] E. Looijenga: On the tautological ring of M 9 . Invent. Math. 121 (1995), 411-419. 
[M-B] L. Moret-Bailly: Pinceaux de varietes abeliennes. Asterisque No. 129 (1985). 



19 



[Ml] D. Mumford: Prym varieties I. In Contributions to analysis (a collection of papers 
dedicated to Lipman Bers), 325-350. Academic Press, New York, 1974. 
[M2] D. Mumford: Towards an enumerative geometry of the moduli space of curves. In 
Arithmetic and geometry (M. Artin and J. Tate, eds.), Vol. II, 271-328, Progr. Math., 
36, Birkhauser, Boston, 1983. 

[NO] P. Norman and F. Oort: Moduli of abelian varieties. Ann. of Math. 112 (1980), 
413-439. 

[01] F. Oort: Subvarieties of moduli spaces. Invent. Math. 24 (1974), 95-119. 
[02] F. Oort: Hyperelliptic supersingular curves. In Arithmetic algebraic geometry (Texel, 
1989) (G. van der Geer, F. Oort, J. Steenbrink, eds.), 247-284, Progr. Math., 89, Birk- 
hauser, Boston, 1991. 

[03] F. Oort: Complete subvarieties of moduli spaces. In Abelian varieties (Egloffstein, 
1993) (W. Barth, K. Hulek, H. Lange, eds.), 225-235, de Gruyter, Berlin, 1995. 
[SV] K.-O. Stohr and J.F. Voloch: A formula for the Cartier operator on plane algebraic 
curves. J. Reine Angew. Math. 377 (1987), 49-64. 

Institutionen for Matematik 
Kungliga Tekniska Hogskolan 
S-100 44 Stockholm, Sweden 
faber@math.kth.se 

Korteweg-de Vries Instituut 

Universiteit van Amsterdam 

Plantage Muidergracht 24 

NL-1018 TV Amsterdam, The Netherlands 

geer@science.uva.nl 



20 



